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(54) ACTION DETECTION SYSTEM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To automatically acquire probability models for 
standard actions through the use of hidden Markov models by observing the 
world through a video camera and other sensors, and to detect an abnormal 
action by inferring the kinds of activities which are currently monitored by 
using the models. 

SOLUTION: In an embodiment, a system consists of image analysis, model 
acquisition, and signal analysis as three constituent parts. In the image 
analysis, respective frames of video are deformed into vectors of a 
sufficient number to represent the operation of an object 18 in front of the 
camera 14 and a series of vectors establishes one 'signal' for each frame of 
the video. In the model acquisition, signals are analyzed so as to obtain 
parameters as to dynamic probability models of the scene in front of the 
camera 14. In the signal analysis, the models are used to classify and/or 
detect abnormality of signals generated speedily by image analysis of new 
video. Consequently, the processing time can greatly be shortened. 
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g tTfttfeW v-^ x A *5 V ^T, 

itfJtfiA^ff t lyhPf-W^T (entropic prio 

.5 rim ±9. ttjtem^xVb^^-rs^gt^e^ 

[f##f2] fJfB^V t-Pf-^-T T (entropic 
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40 1 Merging for Hidden Markov Model Induction)! (B 
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tJ^T, [ft^TlV^V XA£f£a*LT^-5„ Shiro Iked 
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(^^(Construction of Phoneme Models — Model Searc 
h of Hidden Markov Models)] (-C V bit 
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50 [0 0 0 6] 
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[0 0 15] :07/^!JXA|±, «l#^7 7^t, 
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30 fcttflMK^frSrlifc*-*-*. :©7/^"!JX'A©ii!t 
0 A <Dm&X°<DHMMs<7 t—fimcmirZ)^ hot' 
-Kj7°7-f T(entropic prior) t MA Pit Jtl^lIB-fSfc? 
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i±, ±E<0^tfv/i'3 7^7 f /UT'fc5„ JWPJfWv^x 
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##fcft5„ • 7vu=*y Xaii, nry)j a 
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A*l 8 0SSfttTK)^^tli1-5, .=>- =5, M et^SStt 
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[0 0 2 3] ASi 2t*5tt5^-C©SttJ3J:tJ«»f^H: 

•9tfL-5s ±ifi<7)3i> h n t°-ft7°7-f T(entropic pr 
ior)ro*iJfflSrfflL-C h 

T^*mm Lxmn (D^mnmmmn 5 «t 

©trii^ci.- y n 6 5 z t tfmMZtii. 

Oo Z<nm-&{U-i, A® I 8(±g53-W^=.^-2 8(D1E 



4oAEpiSj£tl, #R}4Sj5Sl 2<DJSH$:I* <1~ 5c 
[0 0 2 4] r^TH3^#HB,1-5i:, 4§ffi 1 3 {4, zc 
y b O t°— T(entropic prior) A h- 1/—=^^* 

$*t5 4 5 ja ^mm^momm^f^m 

mz.Wfti-?>^~y b 1 6»a^^H?Tl-5, mw3 

5o 77*3 8(d4oTll^£tL.54?tA 

4 0A[I],A£;h,2>4 5lA SftfcfrSiriM&fflSftS. 
[0 0 2 5] :;t*14 Sr#flB1-5 t , W^tVkoI 

SHit47 u— Aft !c, »ffirt«M*4fcttBMf©»f^Sr 
^■ffc^W^^ bMz.lEz.btiS. 44-eg 
*£ti5 4 5 (A *7*^###&;h,5ft*«:*Mi1-3 
-ii©^ h^t/£5 0 4 6 T*0*$tV§ «t 5fA *7 
^#{-(4, 1^7^3 7*7^7;^!) XJ*<Dmm 

^*7VU£#-CL* 0 t, 4 8T*&^£tl5 4 9 iA if 20 

[0 0 2 6] ^^-CH5S:#|»1-5i:, 

NWr^5A;*7AcD$!;f>{4, mti^^^y^f 

^SrPI±1-5r iiaot, Rlft-r/Ha 7*7VUtfV? 7 30 
^y h n fc°-#J7°7A7(entropic prior) 5 4CQflJffl(C 

*1*B8ii-f5. LfcJiot, 111,^37*7;^ 

[0 0 2 7] ^ hp t°-ift7°7AT(entropic prior) 
14, 5 6T-|21^$^5 4 5 (A ^SC&flbMfflf^ff 

^¥PR5 oiASSM^ttrv^o 

[0 0 2 8] SCHiC^T, *fc, 1*1^3 7*7^ 40 
p e (0.) = 0. e W' loga ' =e-" w 

[0033] x>Fn h'-m^y^ T(entropic prior) 
!40 t 1T"2 0C7)S^«^*1-5 0 &*«Jfc N r:*Ui« 
ft^MfiA, ^->A7£A3, mz.ll, OlC-tybZ 
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tfib^Mm-^ztiz* 

[0 0 2 9] S&fciyjlfcRm-Sfcfcfc:, ^T^« 
A» b AM K^tt T cD JS Iffl^tj AM ©frij*7><D 

e«t*Hfft**©^*5lj3i»fefia:^ h p fcr- 

(D^m&Mnm&^T^^n-r § $? l v y xa 

(A rtifeco^-^Hi, LtfLtfAWfcil^trtSISir 
WiMxhZ 0 rti£>f4, 7o-7t- 1- 1 LTfc5W4 

y-fh. Lxm^-WO ^mxh^nm<DWM^r'7UXh 

[0 0 3 0] ZCDffim^-t&mii. 1^7*3 7^ 
}d *s it 5 fltigft 4 7 * - * (7? m H#H # |c BBi- 5 , 
frfc/^W^tftAft: (EM) T/i^'y XAAfc-5, ^ 
T/U=ryXA(4, t»^77i5^fii LT&iEfc 

ymz>z.k\z£<9 m^cDBtitzffimm&x. 

t-lll1-3^>" hn f-WT*^^ T(entropic prior) t, 
*^*Att (MAP) «UemH-t5»Tfc5. MAPi 

(4, mmt Lx'&iLtzt£\,^<7 zm^-tsmmz 

h<9, Z.tiblt^rfrfrbty<0m&Z.ttfX%Z> o Zti 

4 or, mMxtmi.^t£^ ; f^m^ ] o m znz* 

[0 0 3 1 ] ^>hnt°-^t|»J(CfcV^Tf4, 

7 ^ - ^ (4, * tvW^sJ fc if «Sr(«B L* V n r t i« fe 
>9, Lti5oTi*SiSifiSt5. 0i:/c(4liCfiV^X 
7 ^ - ^ (4, tf>t 4 -iX Ij ;<~ Ztitzim-Oftmiz. 

m^m%m-t(Dx\ tt«tLrstio 0 rws^7i 

(4, ^7^-* ■ 3->hnt-{C>fftT*ffi*li«^W 

^© 4 5 ftto # 9 Ltzftmxmz, r. t ^ -e § 5 „ 
[0 0 3 2] 

(1) 

(2) 

mist, m^ztitz&m*. nmommK^^mm^ 

(A ^>hnt°-^)A7-YT(entropic prior) {to T 
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Z>+fttmmX'hZ> 2/3tl|Xj£1-5 ( , 
ifitH £ ttT V ^ V ^-gi- ft, Wfi» fix — ■ ^ -Sr 
*<7>*££**A£ISLTJ;i9iSv^S*© 

fcliE«^/ < eV^-g-t(i'7 P 7'f T (prior) iiO^fcill 
^ftf^tSr^LTV^c 10 
[0 0 3 4] Lfc^oT, 0 A (Officii, $n{»jft3g# 

ix 6 # x - ^ cofS^W#tt SrlBJfi-f 5 Sf&£ *T-f 3 r 
Pe(0 | a>) oc .Pe(<9) n 6.°" = n 0/ i+Wi 
[0 0 3 9] ^<D-?74T (prior) HE 

/5><9, ^0»-g-}C(±*;m5S*(*« (3) tmCBM*®. 
fl^xV !J ^ KD7°7-f T (prior) frb&btlZbn 

kxh5 0 30 
0 = A-(iogne I " i+9 ' + A|;e l .) 



[0 0 4 3] e, ^?W1-3t, *©S;^e>tb5o 40 
[0 0 4 4] 
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[0 0 3 5] ffiflcfxtf, y°y4T (prior) i±, g 
[0 0 3 6] ±ia»e A <7)l£fflie:<fci9, 0 7 (a) , [3 

7 (b) s 17 (c) immLx&,Txmbfrte£? 
[0 0 3 7] ^btiKit me^&womm, z. 

(D-fy^T (prior) ft*flfc*fltftKl^(K 1 WSfttSriif 
tt, */HBf4 1 / 2 X'h <9 , 3/^°7^-^^COV^Ti±l 

ropic prior) imbtlia --T\ co , Cl'J» 'ft y -f V i 

[0 0 3 8 ] 
[*2] 

(3) 

[0 0 4 0] t°' fr-JT"^ -Y T (entropic prior) 

[0 04 1] MAPt|£Sr#5fc*}Cltt, flllfcKlE, e 
, =l^£5i5l;fRfMot, *f»±«ro«IM 

[0 0 4 2] 
[*3] 

(4) 
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[$C4] 

(6) 



[0 0 4 5] ;it*W- , hwwit»3 

a-cifet), w ( x ) e w < * 1 =x^tti-z>£m%x°h 

Z> 0 it (5) Stfit (6) (4, xyfcf-ftMAPl 



[0 0 4 6] a (3) ©»»H*<0*^fcH:, 

[0 0 4 7] 
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log n e B < + "> = J (e, + co- ) log e o 

= ^(0 log 6? ; + 0)j logtf, -ODi log£Uj + £» f k)g«» ; ) 
N N CO- V 

- \og6 ; - ^ co i log™ + ^<o ; log«j ; 

- -£>(<» || 6)- H(co) 
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[0048] CKDJ; 5 MAPtt£{±, /<y* — 9 ■ 
iVl-cf-H (0) ir, ^7^-^etf-?©+^ 
&$Ef+a> b<nm<D? n^^v hn f-D (tulle) b(D 

[0 0 4 9) ^visDh°-^MJ|±, ^7^-^<t**r 

p{6) piX\8\8,) 

[0 0 5 1 ] ^7^7 (prior) tfULt, 
1 



10 kftX±<Dlg.m&&m'P£ltZ<DX\ 

^IwjfSU^o ±£««fcp (X | 0 \ e s ) 1fiy°y 

■iT (prior) p (e\6, ) [d*5lt^fU#(Cj;oT*S 

[0 0 5 0] 



[ft 6] 



[0 0 5 2] 
[»7] 



io g ^- ^ log p(z 1 0) - log p(z 1 0 \e.) 



(9) 



(1 0) 



[0053] e, 5&s/h£v>ftp>f;f, *»**3S»cftAi- 



-0,. logfl. id,. 



dlogp(X|fl) 

30, 



[0 0 5 4] 
[»8] 



(1 1) 



[0 0 5 5] 



•0, T-fiJoTf-S1-5t, 7ic^5W#f> 



[0057] LfdSot, *f»**o4aEtt*#K:*S: 

[0 0 5 8] iyFDt*-^7"7^7 (entropic prio 

K w -§«i(0^/<y».k)A('+i) 



[0 0 5 6] 
[*9] 



(1 2) 



[0 0 5 9] HMM»W©iy h n t°-W«iIiJ(Cfc 

[0 0 6 0] 
[»1 o] 

(1 3) 



[0 0 6 1 ] at pit, fmti-^^^y^f^ 

(A Tutorial on Hidden Markov Models and Selected A 
pplications in Speech Recognition)] (I EEEe 
i?7 |?2-§\ 257-28 6-<-v\ 198 



9¥) iefc^TRabiner^^ai1-5 <t 5 Id, mi* • 

, t»ft6rtti^ iffc/j;*^ {Pi , , } , = 
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[0 0 6 2] x.>ho¥— fifth Is— ->?ti&<D3:o 

U , »4lf ^cofW^fc 5 r t & 4 19 ^feictftSU $ 
(3) iy h p f-ftl; h W^y^"^ jif:HMM(l 
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(4) iy h n tf-Wl: t- i^-=y^$tifcHMMU, 

[0 0 6 3] MJ 5 V^WSM^V^-C, HMMlCfc^ 
[0 0 6 4] 

[fl 1] 

(14) 
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Lry-x^ V ^(gating) fcWtfjh/TV* 5„ 
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2 as waffle $ Y>\sxhz>*Yy4 t?* 50 
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Tift** C. Wren, A. Azarbayeja 

ni, T. DarrellfcilM. PentlandiF Tpfinder: Af*cO 
^R#H9i§l$(Pfinder: Real-time Tracking of the Huma 
n Body) J (S P I E#$g, |2615|, 1995 

SSKUTV^S. ^ 7='-^(c:MLTf4, #7?^ 

id 5-1 0CO^ ^7-1'7°^ft^TV^fc (1 MJ-<^ h/U 
14, h7^7°(C10V^T [mean, extent, Amean, A 
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h/l^i4, [mean,, mean,, Amean,, Amean,, mass, 
Amass, elongation, ecoentricity]"CtS|5Sc$ixTV N 

[0 0 7 0] fi30 &<D 1 f—i'tf (SGI I n d y C 
am) ^b4HxXnhntc 0 t UWfr b 

iStJU L t # © ffi S y u- a g m fi BiJ l» $ tiZ> t s 

[0 0 7 1 ] W#$tt5 4 5(-, • 7-9\z.fct> 

LfcS$««Sr^rLT*SiJ, ^ h7^7 - T-?\c\tb 
i4«fg5rK57t*!diB$r#(4*i-*J-g-^ f\ frcoMftplcig 
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[0 0 7 2] M/-=y^"tSLT!t ft^SiOO^ 

1 9 0 0 7l/-i^(d^5 .jr 12, 16, 
2 0, 2 5t3£Xfi3 0Vtm<DHMM.<Dx.l/ h n f— h 

k fc * ^ /w£3S 19 flit 5 4 5 KUMtfHfc S 3xfc 0 3&ft 

*#]W#;il©$l*f4g(edge) <Dft V ^tltol %tf± 0 
i§jt£tL7t.„ ^tLiiH7 (a) (Cffl^^tlTV^S^, ■£ 10 
^-CttflFVMIWi h 1) y**fc*J»*5*# 
ftftSr^U ^5V«f4 0f^i£Wit£^1- o WfflwS 

^A»ofc„ hW=.y^liMa t l a b £ ^Kj $ it 5 
SGIR 1 0 0 0 0T6 g>^o fco 

[0 0 7 3] £;h,&<0**©iB*tt&©fc:fci}T?fc5. 

i^hPt-«4H'-=^tt, 0 7 (b) [C^Lfc 

<omLtc 0 ?n-^- h£fc(4^i5s»o«£ 20 
ffSr^urv^s. aBl-^tdiii, 09co^-pi^ffiw 

h a fcf-tfjfc h w-=^^{4S*«J*S»Sri"C»!:*ja, 

3 0 v^<oA»©*tffiSrW^ISiW1-5ri:{c:1-5 0 #tffi5 30 
tt, 5*-* &*T/Wfr£-f -fcwti&wtfSgi ©TO©/-** & 

-To ttllOli, V-f&mL-tSZ.k&m-f, * 
f®9(4, KMT5LfcO^'- K*»&*± 

[0 0 7 4] ^WiC, «©hl- = >^U 0 7 
(c) <Dm^<7/-f"? h]) t/9XKLtS\i*XSLt>1M 

i-wca*.*. eii o\ciF&ftizm%ttmvtm<Dfflfi 

[0 0 7 5] &ts, m&(D%3b*lZW<0W(Di%, 
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[0 0 7 6] Stt*fT»&£ffl'*-5 J t7*A'©i&frBu x. 

0(OT^-t?MlWc 0 (a) hi/ 

--•ytf.?-*, (b) fc&^ftfcWIfer-^ 

(c) steu-c»ttstifciW7*-^, (a) &%-m* 

*&'*7 J* ©iSffJW*'** (power) £&S! LT^ 
5. *©3o©StR^#iSfeofc 0 ( 1 ) 

ishut°-mmfcztitz;<y*-?, (2) vtmmz. 

miE£thtz;<yt-? s (3) ffi^^¥e>^Sttfc51^ 

5„ El 8 (4, r-^Srx^ h-tSfcfelciJ; tj i<— jR{k 

&il*#£iattfc^»£EJ3"J1-5 r fc tstfiEa Lit - 
t^LTV^. ¥^(-^ofc^x><DKiSf4, ^©1 

[0 0 7 7] g^fSt, r*L*-e*LT*fcOI4 N W 
ft lc *i it 5 ^{k^ffig KSft SrfioT, = > t° ^ 

(entropic prior) Srfljffl LX , HMMtB|-f5«jt*5j; 
->^r-A^:f*!4, mftro7-^^x-->a y©^y 

[0078] *mw<Dmm®m^<^frk%ti\ctti-5 

/ c CtW^1-#''f ! PS^1-?)t(73TI4/ e C<, 0iJt LT©^1 

^stir^sr ktf%M%iatm&mt-rx'3b% a #< 
co^fii^^roffiwiift0iji4ail#(DffiHrttfe ^ , m 
f+0»*q[*i J: t/^-ti fc [i]#^g|55j v (c 4 o r w^PS^ $ 

[0 0 7 9 ] 

^»^Bji(i«5ft©j^ffi->^xA{4, u 

/i' S: g » 6^ lc S W-t" 5 tT»J^ ffl -> ^ r A (c fc v ^ T , Bti 

^37* f^mmi- z^Wik, muxMt^k^-y 
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h u t'-^-Zy^TCentropicprior) S-S-^VTiiilHR 
ti-?/u=i 7^-7 A-Sr Y is ■>?■$- Z> - t \c 4 <o , IfjfB 

[0 0 8 0] ro*WK:«5fT»l*a5/^^A 

(entropic priori 0 A OMffcl!), Lfc^otat' 
fflf$R£ tTSSL/c^^l^^^^-rstufBHix-^^^^^ 10 

7v^e>©^7^-*£^*t-5©T\ affiESTJ-fc 

[0 0 8 1 ] Sfc, r©»WJH«Sfi 1 »*U^7 l A 
(4, J*l±RWLfci:*9, lKHEM'-=>^*s, ffifBIB 

7)\ ^> h n t°— #J7° T (entropic prior) Sr^tfOD 
[0 0 8 2] roJBWKWSfTKtfeffl^^A 

tfrtB^* h/^W1EA*(t SWT?, a&iftST 

[0 0 8 3] ::©*Bfc«5fTil|*ttl$'*7 l A 30 

V h ci t°— Kj7°7V T (entropic prior) Sr^tf <D X\ 51 

[0 0 8 4] rojSSC«3fTftfi|(ili/^TA 
(4, «±K8 US: fJfB^Vbne-Wyy^T 
(entropic prior)/)* 0 A O^StfcSW?, H&UST 

^•^#5S«)**^1-5o 40 

[0 0 8 5] r©*wic«5tT»tfta^^^ 
(4, RJJRHUtfcfciJ, S»WtT«i(cP^5StjJBA^ 

[0086] zhi^ zcomm^m^mmtHis^M. 
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[iaiii<z>ffi¥ttK9i] 

[Hi] ^7^x{i D n D i % k&jWctA>;M7©3B«mc 

^^t>tiT^5 At^^m^mxh <o , z »«ffii4t- 

So 

[02] it^A hstf— Kfcffl^oT^SjK ^rttdiz; 

©l&tt^H^ Uc0 1 c7)^S»HT-fc 9 , fcfx^-* 7< 7 
*j it^Bix^/Pa 7^e-xVk4A^' LT^ 
5*»Sr*fflUT. Atf^lst'^-? mmximZft-oX 

[0 3 ] «*«jfi : »<o«* i e7*/WS4-f Btb-vyua 7 

£\ mbtti-^TW^^Ti^roto-^fcS, Hi 

[0 4] i!{t#$T\ ^^SWSsitKff^WSr-^tf 

[15] M8 A ?rft5i>hnt-W7"7^7(e 
ntropic P rior)coj»A(c4t9, Rff***$:«»W-SS' 

[0 6] a^$ti5!iE*iC0 A 7°7-f7 (prior) £ 

Id 4 o TtTbfL5tt**i$K LT* L,fc^7 7T*fc5 0 
[0 7] (a) (4M^-=>^miroKtl-v^='7^T f 
/KD^7^^-7hy 7 ^X(Di?ffe5 0 (b) (4^ 
> hof-WJSSiSrfflV^fc hU-=>i/^cO (a) © 

IJlv/^7 ; tfVW^7^-^-7 h !) S/^^COffl^T 1 

*> 0 . L* ittxff^ t * 5 Xhh 5 
^^7^-^©#I^Srlll*LTl^ 0 ( c )(43i>bn 
k°-W«£4£ffifflLft^hU-^>^© (a) «75|» 
}iv/^7^W/v^-^? b V xcDffi^Xh 
9, ^^•^^7^-^/iSi;*(045(- ; e7/K^§ttTV^ 

[0 8] li'hnb'-WfcM^— =>^**LfcHtb-v 

3 7^xVix4 0 fcH#«S-©tftU<Dfls£& 5 4 < ft 5 c: 

[0 9] 0 7 (c) <©Bti-77W='7' : tT : VK7)jf#-^ h 

ys/^^i(i^4fi^tifc7P-ft-Ft*fct), 3=->-h 
[010] 0 1 fc4t>*0 2 coSScOAWSW^Bf fflW 
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